On the Total Reward Variance for Continuous-time Markov Reward Chains
نویسنده
چکیده
As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for continuous-time Markov reward chains with finite state spaces. The results correspond to discrete-time results. In particular, the variance growth rate is shown to be asymptotically linear in time. Expressions are provided to compute this growth rate.
منابع مشابه
Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains
This paper deals with denumerable-state continuous-time controlled Markov chains with possibly unbounded transition and reward rates. It concerns optimality criteria that improve the usual expected average reward criterion. First, we show the existence of average reward optimal policies with minimal average variance. Then we compare the variance minimization criterion with overtaking optimality...
متن کاملDistributions of Reward Functions on Continuous-time Markov Chains
We develop algorithms for the computation of the distribution of the total reward accrued during [0, t) in a finite continuous-parameter Markov chain. During sojourns, the reward grows linearly at a rate depending on the state visited. At transitions, there can be instantaneous rewards whose values depend on the states involved in the transition. For moderate values of t, the reward distributio...
متن کاملAggregation Methods for Markov Reward Chains with Fast and Silent Transitions
We analyze derivation of Markov reward chains from intermediate performance models that arise from formalisms for compositional performance analysis like stochastic process algebras, (generalized) stochastic Petri nets, etc. The intermediate models are typically extensions of continuous-time Markov reward chains with instantaneous labeled transitions. We give stochastic meaning to the intermedi...
متن کاملSecond Order Optimality in Transient and Discounted Markov Decision Chains
Abstract. The article is devoted to second order optimality in Markov decision processes. Attention is primarily focused on the reward variance for discounted models and undiscounted transient models (i.e. where the spectral radius of the transition probability matrix is less then unity). Considering the second order optimality criteria means that in the class of policies maximizing (or minimiz...
متن کاملReward Variance in Markov Chains: A Calculational Approach
We consider the variance of the reward until absorption in a Markov chain. This variance is usually calculated from the second moment (expectation of the square). We present a direct system of equations for the variance, involving the first moment (expectation) but not the second moment. This method is numerically superior to the calculation from the second moment.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2006